THE MOST FREQUENT PEAK SET OF A RANDOM PERMUTATION 
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Abstract. Given a subset S C P, let P(S; n) be the number of permutations in the symmetric group 
of {1,2, . . . ,n} that have peak set S. We prove a recent conjecture due to Billey, Burdzy and Sagan, 
which determines the sets that maximize P{S; n), where S ranges over all subsets of {1, 2, . . . , n}. 



1. Introduction 

Given a permutation ct = cti (T2 ■ ■ • f „ in iS„, the symmetric group of the set [n] — {1, 2, . . . , n}, the 
up-down sequence of a is the word F{a) = z\Zi ■ ■ ■ z„_i with letters in {+, — defined by = — if 
I is a descent of a (i.e., Ui > cn+i) and Zi — + if i is an ascent of a (i.e., ai < (Ti+i). The enumeration of 
permutations with a given up-down sequence is a traditional topic of classical combinatorics which has 
received considerable interest since the end of the 19th century (see e.g. jU [2j [3l lU [9l [10] ) . A classical 
result in this topic that has been proved several times during the last century (see e.g. [5J|P1[TD] but the 
list is not exhaustive) is that the maximum of /3{w), the number of permutations of [n] with up-down 
sequence w, where w ranges over all up-down sequences of length n — 1, occurs when w is an alternating 

sequence, i.e. w = -\ 1 1 — ••• or w — — 1 \ 1----. A refinement of this maximization 

problem which consists in maximizing /3(w) where w ranges over all up-down sequences of length n 
with a fixed number of alternating runs, has been conjectured by Gessel and was solved by Ehrenborg 
and Mahajan ■A]. 

A related maximization problem has recently arisen in the work on the peak statistic by Billey, 
Burdzy and Sagan [T]. Recall that, given a permutation cr = cti (T2 • • • cr„ in 5„, an index i is called a 
peak of cr if ai^i < ai > (Ti+i. The peak set is defined to be 

VS[a) = {z : i is a peak of cr}, 

and given a set S of integers, we set 

V{S:n) = {a e 5„ : VS{a) = S}, 

and V{S:n) = 4fP{S;n). For instance, if cr = 26 5143, we have PS{a) = {2,5} and thus cr e 
■p({2,5};6). The numbers P(iS';n) have been the center of the work of Billey, Burdzy and Sagan [l] 
and a conjecture on the sets S that maximize P{S;n) among all subsets of [n] has been proposed. The 
main purpose of the present paper is to solve this conjecture. To present their conjecture, it is more 
convenient to use the notion of peak-composition. 

Suppose that the peak set of ct G iS„ is VS{a) = {ii < Z2 < • • • < U-}. Then, the composition 
pc(cr) = (ci, C2, . . . , Cfe+i) of n defined by Cj = ij — with io = and ik+i = n is called the 
peak-composition of cr. If c is a composition of we set 

Vic) = {cr e Sn ■■ pc(cr) = c}. 
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and P(c) = #P(c). For instance, if 2 6 5 1 4 3, we have pc(ct) = {2,3,1} and thus a £ P(2,3,l). 
Note that if 5 = {ii < 22 < • • • < ik} is a subset of [n] then n) = ■P(c) where c is defined as above. 
For instance, we have 7^({2, 5}; 6) = 7^(2, 3, 1). The size of a composition is the sum of its parts. In the 
sequel, we say that a composition c is maximal if P(b) < P(c) for any composition b having the same 
size as c. The fohowing resuh has been conjectured by Bihey, Burdzy and Sagan. 

Theorem 1.1. For n>l, let C*{n) denote the set of maximal compositions of n. For £ >2, we have 
. C*(3^) = {(3^),(4,3^-2,2)}, 

• C*{3£ + 1) = {(3", 2, 3*, 2) : s > 1, t > 0, s + t = f - 1}, 

• C*(3^ + 2) = {(3^2)}. 

Equivalently, for n>Q, the sets that maximize P(S';n) are 

• ifn = (mod 3), {3, 6, 9, . . .} n {1, 2, . . . n - 1} and {4, 7, 10, . . .} n {1, 2, . . . n - 1}, 

• ifn = l (mod 3), {3, 6, 9, . . . , 3s, 3s + 2, 3s + 5, . . .} n {1, 2, . . . n - 1} with 1 < s < [f J - 1, 

• ifn = 2 (mod 3), {3, 6, 9, . . .} n {1, 2, . . . n - 1}. 

In the above theorem, exponent just indicates iteration. For instance, (3^,4) is for (3,3,4). We will 
also determine the cardinality of V{c) when c is a maximal composition. The expressions obtained are 
extremely simple. 

Theorem 1.2. Suppose n > 6 and c G C*{n). Set £ — [^J . Then, we have, 

• ifn = (mod 3), P(c) = ^S^-^nl, 

• ifn = l (mod 3), P(c) = fs^-^^n!, 

• ifn = 2 (mod 3), P(c) = 3-'^?i!. 

Our proof of Theorem 11.11 is essentially based on some comparison lemmas presented in Section 2 
and a counting formula for the number P(c) which is efficient when c contains only short patterns of 
elements distinct from 3. In Section 3, we show that a maximal composition can not have a part greater 
than 4. In Section 4, we provide further patterns that a maximal composition must avoid. In Section 5, 
we preset our counting formula from which we deduce Theorem 11.21 Finally, the proof of Theorem 11.11 
is completed in Section 6. 

2. Comparison Lemmas 

In this section, we provide some comparison lemmas. We first set up some additional terminology. 
The concatenation of two compositions Ci and C2 is denoted by Ci © C2. For instance, (4, 3, 2) © (2, 3) = 
(4, 3, 2, 2, 3). We assume the existence of an empty (with part) composition e such that c©e = e©c = c. 
We say that a composition c is admissible if V{c) ^0. It is not hard to prove the following result. 

Fact 2.1. A composition c = (ci, . . . ,Ck) is admissible if and only if k = I or ci, . . . ,Ck-i > 2 and 
Ck > 1. 

Given a composition c = (ci, C2, . . . , Ck), we set r'c = c if = 1 and r'c — {ck + 1, Ck-i, . . . , C2, ci — 1) 
if fc > 2. Considering the reverse of a permutation, we obtain the following result. 

Fact 2.2. For any composition c, we have P(c) = P(r'c). In particular, c is maximal if and only if r'c 
is maximal. 
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For a, 6 e P and c a composition of an integer n > 1, we let 

Inia^b{c) {cr e 7'(c) : a ^ ai, cr„_i < (T„ b}, 

Inta^b{c) = {a e V{c) : a = ai> (72, CTn-i < cr„ = b}, 

and Ini.^b{c) — IJ"^^ ^"-*a.b(c)- We set Iniaj, = ^Inia.b, Ini.,b = jflni.^b and Inta.fj = 4/^Inta,b- We also 
let 1 + c denote the composition of n + 1 obtained from c by increasing the first part of c by 1. So, if 
c (ci, . . . , Ck), 1 + c = (ci + 1, C2, . . . ,Cfc). For instance, 1 + (2,4, 3, 2) = (3,4, 3, 2). 

If X = (xi, X2, . . . , Xfc) and y — {yi,y2, ■ ■ ■ ,yk) are two sequences of the same length, we write x < y 
if Xi < yi for 1 < i < fc and x 7^ y. 

Proposition 2.3. Let c be a composition of an integer n > 1. Suppose that there exists a composition 
c' of n such that 

(1) Inta^b(l + c) < Inta,6(l + c') for all a,b ^ [n + 1] and there are two indices u,v such that 
Int„^„(l + c) < Intu,'u(l + c'); then, for any nonempty compositions a and b such that a © c © b is 
admissible, 

P(a © c © b) < P(a © c' © b), 

and thus, a © c © b is not maximal. 

(2) (lm.^b{c)) < (ini.^b(c')) !<{,<„•' tf^^f^, for any nonempty composition b such that c © b is 
admissible, 

P(c©b) < P(c'©b), 

and thus, c © b is not maximal. 

To simplify the readability of the proof of the above result, we first state a preliminary lemma. In 
the sequel, a permutation r = ti • • • t„ of a set S' C P with cardinality n is said to be order-isomorphic 
to a permutation cr = cri • ■ • (T„ in Sn if for 1 < i < j < n, Ti < Tj is equivalent to ct^ < aj. 

Lemma 2.4. For i — 1,2, 3, let c'*' be a composition of a positive integer Ui such that c^^"^ ©c*-^-* ffic^'^^ 
is admissible. Suppose Int„ „(1 + c*^^^) > 0, then for each permutation r G Int^ y^l + c*^^)) there exists 
a permutation a in V{c'^^^ © c*^^^ © c^'^^) such that cr„iCr„i+i • • • ct„j+„2 is order isomorphic to r. 

Proof. We first choose two permutations 7 G /ni. ,^(0*^^)) and (3 G /ni. „3+i (r'(l + c'^^)) (this choice 
is always possible since, as it is easily seen, /nt.^„(c) 7^ for any composition c of n). Then set 

Cl, ■ • ■ , CTni-l = 71, . . . , 7ni-l (resp., ani+n2+n3, CTrii +n2+n3 - 1 1 • • ■ : 0'ni+ri2 + l = /?! — 1, . . . , /3„3 +ni — 

1). We then set fT„j, cr„j+i, . . . ,cr„j+„2 = ri+rii+ris-l, T2+ni+n3-l, . . . , Tn^+i+rii+na-l. It is easily 
checked that the permutation a — ai - ■ ■ ct„j+„2+„3 is in V{c^^^ © c^^^ © c^^^) and (t„jCT„j+i • • • ct„j+„2 
is order isomorphic to r. □ 

Proof of Proposition \2.SX (1) Let a and b be two compositions of positive integers r and s such 
that a © c © b is admissible. By our hypothesis, we can consider a family of injections (0c. d) i<c d<„+i ' 
<f>c,d ■ Iritc^di^ + c) I— > + such that „ is not surjective. Then, consider the function F which 

associates to a permutation a G P(a©c©b) the permutation a' defined by a'l, . . . , (T^_i = Ui, . . . , cr^^i, 
"'r+n+ii • • • I '^r+n+s = cTr+ri+i, ■ • • , cr^+n+s, and a'^a'^^i ■ ■ • (7^+„ is the unique permutation of the set 
{(Tr,(7r+i, . . . , iTr+n} which is Order isomorphic to </'ri,r„+i (■''), where r is the permutation of [n + 1] 
which is order isomorphic to arCTr+i • ■ • fJr+n- It is easily checked that F is a well-defined function from 
■p(a © c © b) to P(a © c' © b) which is injective but, by Lemma [2?4l not surjective. 

The proof of (2) is easier than and very similar to the proof of (1), and so is left to the reader. □ 

A simple but important consequence of the above result is given in the next section. 
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3. The largest part in a maximal composition 

Proposition 3.1. Let c = (ci,...,Cfe) be a composition of n. If k = 1 and n > 5, then c is not 
maximal. If k > 2 and there exists i, 1 < i < k — 1, such that Ci > 5 or Ck > 4, then c is not maximal. 

The above result is an immediate consequence of Proposition [2731 and the two following lemmas. 
Lemma 3.2. For n > 3, we have 

(la) Int„,a(n) = Inta,„(n) = 2^^-^ if2<a<n-l, 

(lb) Inta,f,(n) =0 if a ^ n and b ^ n, or a — 1, or b — 1; 

(2a) Inii_„(n) = 1 and Inii_h(n) =0 ifb<n, 
(2b) IniQ,6(n) = Inta,b(n) ifa^l. 

Proof. (1) Suppose 2 < a < n — 1. A permutation a in 5„ is in Intn,a{n) if and only it writes 
a — nwi\w2a where wi is decreasing and W2 is increasing. There are 2"~^ choices for wi and each 
choice of wi determines uniquely W2- The second assertion is left to the reader. 

(2a) Clearly, for n>2, Inii^n{n) = {12 • • • n}. If & < n and a G Sn with (Ti = 1 and cr,i = b, then a 
has at least one peak (the index i such that ai = n) and thus pc((t) ^ (n), whence /mi,f,(n) = 0. 

(2b) By definition, for all integers a, 6, we have Inta,b{n) C Inia,b{n). If 6 = 1, Inia,b{n) = 0. 
Suppose a 7^ 1, 6 7^ 1 and a G Inia,b{n). If cr ^ Inia,b{ri) (i.e., if cti < a2), the greatest integer p such 
that (Ti < (72 <•••< CTp is less than n — 1 since ai = 1 for some index z such that 2 < i < n — \. This 
implies that p is a peak of cr and thus pc(cr) ^ (n). Consequently, Inia.b{n) C Inta.biji). □ 

Lemma 3.3. For n > 5, we /laue 

(la) Int„,„_i(3, n - 3) = Int„_i,„(3, n - 3) = (n - 4)2"-4, 

(lb) Int„,a(3, n - 3) = (n - 2 - a)2"-2 + ^(^a > 3)(a - l)2'^-3 /or 2 < a < n - 2, 
(Ic) Inta,„(3,n-3) = Inta,„-i(3, n - 3) + (a - 1)2"-^ for2<a<n-2; 

(2) Inii,„(3,n-3) > 1. 

Proof, (la) Clearly, Int„_„_i(3, n — 3) = Int„_i_„(3, n— 3). A permutation cr in iS„ is in /n<„^„^i(3, n — 3) 
if and only it writes cr = nxkwiiw2n — 1 where wi is decreasing and W2 is increasing, 1 < a; < fc, 
l<£<k<n — 2 and all the letters of Wi and W2 are greater than £. It is easily checked that for each 
k, there are {k-l)2^'^ such permutations. Thus, Int„,„_i(3,n-3) = J^kZU'^ - '^)'^''~^ = ("-4)2"-^. 

(lb) Suppose 2 < a < n — 2. Clearly, if cr S /ni„_a(3, n — 3), then we have — n — 1. We classify 
the permutations a in /7it„_„_i(3, n — 3) according to the value of cr2. If a < fc < n — 2, then there are 
2"'~'^ choices for the permutations a in Intn^a{S,n — 3) with 02 = k. If 1 < fc < a and a > 3 (resp., 
a — 2), then there are 2"""^ (resp., 0) choices. 

(Ic) Suppose 2 < a < n — 2. We classify the permutations cr in Inta^ni'^,n — 3) according to the 
value of cr„_i. If cr„_i = n — 1, then there are Inta_„_i(3,n — 3) such permutations. If cr„_i 7^ n — 1, 
then era = n — 1 and there are a — 1 choices for cr2. For each choice of cr2, there are 2"~^ corresponding 
permutations. 

(2) It suffices to observe that {124356---n,134256---n}C /mi,„(3,7i - 3). □ 
Proof of Provosition 1 3. H (1) Suppose ffi'st fc > 2. It follows from Lemmas 13.21 and 13.31 that 
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• for n > 6 and all a,b £ [n], we have luta,b{n) < IntQ^b(3, n — 3), and l'nt2,n{n) < Int2,n(3, n — 3), 

• for n > 5 and all b G [n], Ini.,h(rt) < Ini._f,(3,ri — 3), and Ini.,„(r7,) < Ini.,„(3,n — 3). 

Consequently, by Proposition 12.31 if n > 5, then for any nonempty compositions a and b, the com- 
positions a © (n) © b and (n) © b are not maximal. In other words, if c is maximal then Ci < 4 for 
i = 1, . . . , fc — 1. This, combined with Fact 12.21 implies that if c is maximal, Ck < 3. 

(2) If /c 1, then a simple counting (see the proof of the above lemmas or [1]) shows that P(n) = 2"^^ 
and P(3, n - 3) = (("2 ^) - 1)2"-^, from which we deduce that P(n) < P(3, n - 3) for n > 5. □ 

4. Some forbidden patterns in a maximal composition 

The 3-factorization of a composition c is the (unique) factorization of c as 

c = c(°)©(3)fflc(i)©---©(3)©cW 

where k is the number of parts in c equal to 3 and the compositions c*^'-' (possibly empty), called factors, 
have no part equal to 3. For instance, (4, 4, 3, 2, 4, 2, 3, 3, 2, 1) = (4, 4) © (3) © (2, 4, 2) © (3) © (3) © (2, 1). 

The purpose of this section is to prove the following result. 

Proposition 4.1. Suppose c is a maximal composition with at least 3 parts. Then, 

(4.1) c = a;offl(3)ffia;iffi---©(3)ffla;fc 

for some k > 1 and some sequence of compositions {xi)i such that 

xo e {e, (4)}, Xk e {e} U {(2^) : s > 1}, and, for t = 1, . . . ,k - 1, x, e {e} U {(2^), (4*) : s, t > 1}. 

Clearly, in view of Proposition 13.11 and Fact 12. 1[ the above result is immediate from the following 
lemma. 

Lemma 4.2. For any nonempty compositions a and h, 

(1) if c £ {(2,2), (2, 3), (2, 4)}, the composition c © b is not maximal; 

(2) i/c e {(2, 1), (3, 1), (4, 1)}, the composition a© c is not maximal; 

(3) «/ c G {(2, 4), (4, 2)}, the compositions c © b, a © c and a © c © b are not maximal; 

(4) the composition (4, 4) © b is not maximal. 

Proof. For simplicity, if c is a composition of n, the sequence (ini.^b(c)) ^^^^^^ will be denoted by Tc. 
Using a computer algebra system, it is easy to obtain 

T(2,2) = [0,1,2,2], T(4) = [0,1,2,4], T(2, 3) = [0, 2, 4, 6, 8], T(3, 2) = [0, 3, 6, 8, 8], 
T(2,4) = [0,3,6,10,16,24], T(4, 2) = [0,7,14,20,24,24], T(3,3) = [0,8,16,24,32,40], 
T(3,4) [0,15,30,48,72,104,144], T(4, 3) = [0,24,48,72,96,120,144], 
T(4,4) = [0,55,110,172,248,344,464,608], T(3,2,3) = [0,96,192,288,384,480,576,672]. 
from which it results that 

T(2, 2) < T(4), T(2, 3) < T(3, 2), T(2, 4) < T(4, 2) < T(3, 3), T(3, 4) < T(4, 3), T(4, 4) < T(3, 2, 3). 

By Proposition [23i;2), these inequalities imply that if c G {(2,2), (2,3), (2,4), (4,2), (3,4), (4,4)}, the 
composition c © b is not maximal for any b 7^ e. By Fact 12.21 this can be reformulated as follows: if 
c G {(2, 1), (3, 1), (4, 1), (2, 3), (4, 2), (4, 3)}, the composition a © c is not maximal for any a 7^ e. 
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a\b 


2 


3 


4 


5 


6 


7 


2 


[0, 0] 


[0, 0] 


[1,2] 


[2, 4] 


[3, 6] 


[3, 8] 


3 


[0, 0] 


[0, 0] 


[2, 4] 


[4, 8] 


[6, 12] 


[ 6, 16] 


4 


[1,2] 


[2, 4] 


[0, 0] 


[ 6,12] 


[ 10, 18] 


[ 10, 24] 


5 


[2,4] 


[4, 8] 


[ 6, 12] 


[0, 0] 


[ 16, 24] 


[ 16, 32] 


6 


[4, 6] 


[ 8, 12] 


[ 12, 18] 


[ 16,24] 


[0, 0] 


I 24, 40] 


7 


[7, 8] 


[ 14, 16] 


[ 20, 24] 


[ 24,32] 


[ 24, 40] 


[0, 0] 



Table 1. The values [Inta,b(5, 2), Inta,6(4, 3)] 



Thus, to conclude our proof, it just remains to check that if c G {(2,4), (4,2)}, the composition 
a © c © b is not maximal for any a 7^ e and any b ^ e. Again, by using a computer algebra system, we 
can explicitly compute the array ([Inta,b(5, 2), /nta_f,(4, 3)])^^^ ^^^^ (see Tabled]) from which it results 
that for all integers a, b we have Inta,b(5, 2) < IntQ_b(4, 3) and {at least) one of these inequalities is strict. 
By Proposition 12 ■3f 1 ) . this implies that a © (4, 2) ffi b is not maximal for any a ^ e and any b ^ e. By 
Fact 12.21 the same is true with (4, 2) replaced by (2, 4). □ 

Before we end this section, we provide further patterns that a maximal composition must avoid. 
Lemma 4.3. For any nonempty compositions a and h, 

(1) if c £ {(4, 3, 2), (4, 3, 4), (3, 2, 3, 2), (3, 3, 2, 3, 2)}, the composition c © b is not maximal; 

(2) «/ c G {(2, 3, 3), (4, 3, 3), (2, 3, 2, 2), (2, 3, 2, 3, 2)}, the composition a © c is not maximal. 

Proof. By Fact 12.21 the two assertions are equivalent. So it suffices to prove (1). Using a computer 
algebra system, we obtain 

T(4, 3, 2) = [0, 504, 1008, 1488, 1920, 2280, 2544, 2688, 2688], 
T(3, 3, 3) = [0, 560, 1120, 1680, 2240, 2800, 3360, 3920, 4480], 

T(4, 3, 4) = [0, 9072, 18144, 27720, 38304, 50376, 64368, 80640, 99456, 120960, 145152], 
T(3, 2, 3, 3) = [0, 16128, 32256, 48384, 64512, 80640, 96768, 112896, 129024, 145152, 161280] , 
T(3, 2, 3, 2) = [0, 2688, 5376, 7968, 10368, 12480, 14208, 15456, 16128, 16128], 
T(4, 3, 3) = [0, 2688, 5376, 8064, 10752, 13440, 16128, 18816, 21504, 24192], 
T(3, 3, 2, 3, 2) = [0, 887040, 1774080, 2644992, 3483648, 4273920, 4999680, 5644800, 6193152, 
6628608, 6935040, 7096320, 7096320] , 
T(4, 3, 3, 3) = [0, 887040, 1774080, 2661120, 3548160, 4435200, 5322240, 6209280, 7096320, 
7983360, 8870400, 9757440, 10644480] , 
from which it results that 

T(4,3,2)<r(3,3,3), r(4, 3, 4) < r(3, 2, 3, 3), T(3, 2, 3, 2) < T(4, 3, 3), r(3, 3, 2, 3, 2) < T(4, 3, 3, 3). 
This, by Proposition 12. 3r 2). gives the first assertion. □ 

We can even go further with the same method but it will be more convenient and even more simpler 
to use the tools presented in the next section. 
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5. A COUNTING LEMMA 

The purpose of this section is to provide an efhcient counting formula for the number P(c) when 
c is a composition in which the factors of its 3-factorization have a small size. In the sequel, we let 
|c| denote the size of the composition c. The size of the empty composition is given by |e| = 0. The 
following is the key result in this section. 

Lemma 5.1. Let a and b two compositions. Ifh^e, then we have 

(5.1) P(a©(3)©b)= (^'^' + '^' + ^)p(a©(l))P((2)®b). 

Proof. Given a composition c of a positive integer n and a finite subset F C P with cardinality #F = n, 
we let V{c\F) be the set of permutations of the set F having peak set equal to F. With this terminology, 
we have V{c) =V{c] [n]). 

Let r and s be the sizes of the compositions a and b, and let 

i?a,b = {(7,/3) :7eP(a©(l);5), /3 e 7'((2) © b ; T), 5 U T - {1, . . . , r + .s + 3}}. 

Note that #i?a,b is equal to the right-hand side of (|5.1|) . So, to prove (|5.ip . it suffices to present a 
bijection between i?a,b and 7'(a © (3) © b). This is quite easy. 

Consider the concatenation function P : i?a.b ^ i^r+s+a defined by r(7,/3) = 7/3. It is clear that P 
is well defined, injective and r(i?a,b) 2 ^(a © (3) © b). We now check that r(i?a,b) C P(a © (3) © b). 
Suppose (7 = 7/3 with (7/3) G i?a,b- To prove that cr = 7/3 is in "P (a © (3) © b), it suffices to prove that 
r + 1 and r + 2 are not peaks of a which is immediate since cr,. = 7,. > 7^+1 = cr^+i (r is a peak of 7) 
and ar+2 — Pi < 1^2 ^ cr^+a (2 is a peak of /3). □ 

Repeated applications of Lemma [5TT] leads to the following result the proof of which is omitted. 
Proposition 5.2. Let (c^*'')Q<j<j, he a sequence of compositions such that c*^*^) ^ e. Then, we have 

,5^2) P,c<«> e ,3) e c<.> e ,3) . . ^ » c«) . (^,„, ^^.^^ |St„ Ic<«l + a ^^^^ ^ ^^ 

X P(c(°) © (1)) ( W P((2) © cW © (1)) ) P((2) © c«). 



In the sequel, we will use the following formulas: 

P(2,l) = 2, P(3,l) = 8, P(4,l) = 24, P(2,2,l) = 16, 
(5.3) P(2,2) = 8, P(2,3) = 24, P(2,2,2) = 96, P(2,4) = 64, 

P(2,2,l) = 16, P(2,2,2,l) = 272, P(2, 4, 1) = 288. 

The following non maximality criterion is immediate from the above result. 

Proposition 5.3. Suppose c = c(") © (3) © c^^' © (3) • • ■ © c^*-') with k >1 and c^*-') ^ e. If there exists 
a composition b such that 

(1) |b| = |c(^)| for some index i, 1 < i < k - 1, and P((2) © cW © (1)) < P((2) © b © (1)), then c 
is not maximal; 

(2) |b| = IcC') I and P((2) © c^'^)) < P((2) © b), then c is not maximal. 
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Corollary 5.4. Suppose c is a maximal composition with at least 3 parts and with 3-factorization 



Proof. By Proposition 14.11 we can assume that c = a;o © (3) ® © • • • © (3) © a;^ with fc > 1 and 
xo e {e, (4)}, Xk e {e} U {(2^) : s > 1}, and x, G {e} U {(2^), (4*) : s, i > 1} for i = 1, . . . , fc - 1. 

Using a computer algebra system, we obtain 

T(2,2,2,2) = [0,61,122,178,224,256,272,272], T(2,3,3) = [0,80,160,240,320,400,480,560] 

T(2, 4, 4) = [0, 889, 1778, 2726, 3792, 5032, 6496, 8224, 10240, 12544], 
T(2, 3,2,3)== [0, 1792, 3584, 5376, 7168, 8960, 10752, 12544, 14336, 16128] , 

from which it results that T((2) © (2,2,2)) < T((2) © (3,3)) and T((2) © (4,4)) < T((2) © (3,2,3)). 
Moreover, it is easily checked that P((2) © (2, 2, 2)) < P((2) © (3, 3)). These inequalities combined with 
Propositions[23i;2) andOimply that Xk £ {e, (2), (2, 2)}, and x, e {e, (2), (2, 2), (4)} for i = 1, . . . , fc-1. 

To complete our proof, it remains to prove that if c = a© (3) © (2, 2) © (3) ©b, then c is not maximal. 
Suppose b = e, then r'c — (4,2) © u for some composition u ^ e. Then, by Lemma [4.21 and Fact 12. 2[ 
r'c and c are not maximal. If b 7^ e, it results from (15.21) and (15. 3p that 



Before we end this section, we prove Theorem II. 21 It is convenient to present a slightly more general 
result than Proposition 15.31 which is easily derived by an appropriate specialization in Proposition 15.21 
and use of the relation P(2, 1) = 2. 

Proposition 5.5. for fc > 1, £1, . . . , 4 > 1 and cC'+i) 7^ e, we have 



(5.4) c = © (3) © a:i © • • • © (3) © a:fc. 

Then, fc > 1 and the sequence of compositions {xi)i satisfies 

xo e {e, (4)}, Xk e {e, (2), (2, 2)}, and x, e {e, (2), (4)} for i 



1 



. . . ,fc - 1. 



(5.5) 



P(a©(3)©(2,2)©(3)©b) _ P((2) © (2, 2) © (1)) _ 272 
P(affi(3)ffi(4)©(3)©b) ~ P((2)©(4)©(1)) ~ 288 



This concludes the proof. 



□ 



(5.6) 



P(c(i) © (3^0 © c(2) © (3^^) © • • • © cC^) © (3^'=) © cf'^+i)) 

_ (3(4+---+4)+E';i 




Corollary 5.6. For i > 2, s,m > 1, t > 0, we have 



(5.7) P(30 = P(4, 2) = i32-^(3^)!, 

(5.8) P(3^ 2) = 3-^(3^ + 2)!, 

(5.9) P(3^ 2, 3") = P(4, 3™-\ 2, 3"-\ 2) = ^3'''-"'{3s + 3m + 2)!, 

(5.10) P(3^ 2, 3*, 2) ^ P(3*+S 2, 3"^\ 2) = |3"""*(3s + 3t + 4)!, 

(5.11) P(4,30 = ^32-^(3^ + 4)!. 
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Proof. Specializing (|5.6p at k — I, c'^^ ~ e, c^^^ — (3) and ii ^ i — 1 gives 

^(^')=G)' P(^©(l))P((2)®(3)), 

which simpHfies to ()5.7p by using ()5.3p . Similarly, specializing (|5.6p at A; = 2, c^^^ = e, c'^^ = c^'^^ = (2), 
^1 = s and £2 — t gives 

P(3^ 2, 3*, 2) = ' I^i±|LLi):p(, e (i))P((2) ® (2) © (i))P((2) e (3)), 

which simplifies to (|5.10p by using (|5.3p . The proof of the other assertions are left to the reader. □ 

6. Proof of Theorem 11.11 

The purpose of this section is to complete the proof of Theorem ll.il We shall use the following result 
which is a direct consequence of Proposition [521 

Proposition 6.1. Let (c'-*'')Q<j<;j. be a sequence of compositions such that c^'^^ =/= e. Then, for any 
permutation a £ Sk-i, we have 

(6.1) P(c(") © (3) © c(i) © (3) © • ■ • © cf'^)) = PCcf") © (3) © c('^i) © (3) © • • • © c^^"-'^ © (3) © c^''^). 

Combining Corollarv 15 .41 with the above result leads to the following result. 
Corollary 6.2. Let c = (ci, . . . ,Cfc) be a maximal composition with fc > 3. 

(1) If ci = 3, then c has no part equal to 4- 

(2) If Ci = 4, then c has only one part equal to 4- 

(3) If ci = Cfc = 3, then c has at most one part equal to 2. 

(4) If Ci =3 and Ck ='2, then c has at most two part equals to 2. 

(5) // Ci — 4, and c has a part equal to 2 then c = (4, 3**, 2*) for some s,t > 1, t < 2. 

Proof. By Corollarv 15.41 we can suppose that c = xq © (3) © xi ffi • • • ffi (3) © Xs for some s > 1 and 
some sequence of compositions {xi)i such that xq £ {e, (4)}, Xg £ {e, (2), (2,2)} and Xi £ {e, (2), (4)} 
for i = l,...,s — 1. 

(1) Suppose ci — 3 and q = 4 for some i > 2 (i.e., xq = e and Xj = (4) for some j with 1 < j < s). 
By (16. ip . we can assume that xi = (4) (i.e., C2 =4). In this case, r'c = (4, 2) © u for u 7^ e and thus, c 
is not maximal by Lemma 14.21 This contradicts our assumption. Thus 7^ 4 for 2 < i < k. 

(2) Suppose ci = 4 and q = 4 for some i > 2 (i.e., xq = Xj = (4) for some j with 1 < j < s). 
By (|6.ip . we can assume that xi — (4) (i.e., C3 = 4). In this case, c = (4, 3, 4) © b for some b 7^ e, and 
thus, c is not maximal by Lemma 14.31 

(3) Suppose ci — Ck — >i and c has at least two parts equal to 2. By (j6.ip . we can assume that 
Xi = X2 ^ (2). In this case, c — (3,2,3,2) © b for some b ^ e, and thus, c is not maximal by 
Lemma 14.31 

(4) Suppose ci — 3, Ck = 2 and c has at least three parts equal to 2. There are two cases: Xg = (2, 2) 
and Xs — (2). If Xs — (2,2) (resp., Xg — (2)), by (|6.ip . we can assume that Xg^i = (2) (resp., 
Xs-2 = Xs-i = (2)). In this case, c = a © (2, 3, 2, 2) (resp., c = a © (2, 3, 2, 3, 2)) for some a ^ e. In 
both cases, c is not maximal by Lemma l473l 

(5) Suppose ci — A and Xj = 2 for some j with 1 < j < s. By (|6.ip . we can assume that xi — (2). 
In this case, c ~ (4, 3, 2) © b for some b 7^ e, and thus, c is not maximal by Lemma [4.31 □ 
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Combining Corollaries 15.41 and 16.21 shows that 

(6.2) if c is a maximal composition with at least three parts, then 

c G {(3^, (4, 3^, (4, 3^-2, 2), (4, 3^"^, 2, 2), (3^ 2), (3^ 2, 2), (3^ 2, 3*, 2) : i>2, s>l, t> 0}, 

which is very close to Theorem 1 1.1 1 On the other hand, using Fact 12.21 and Corollary 15.61 we see that 
for s > 1, t > 0, fc > 1 and < j < fc - 1, £ > 2, we have 

P(4,3^-^2) = P(30, P(4,3'') < P(3^2,2), P(3", 2, 3*) < P(3"+*, 2), 

(6.3) P(4,3^"2^2,2) < P(3^2), P(3''"^ 2, 3^ 2) = P(3^ 2, 2). 

Combining (|6.2p with (16. 3p immediately implies that Theorem 11.11 is true for compositions with 
at least three parts. The validity of Theorem 11.11 for compositions with two parts (it suffices, by 
Proposition 13.11 to consider compositions with parts less than 5) can be treated by computer or by 
hand, and so, it is left to the reader. 

Acknowledgements. The author would like to thank Christian Krattenthaler for fruitful discus- 
sions and its encouragement during the preparation of this paper. 
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